Image predictive encoding and decoding system

ABSTRACT

In image predictive encoding according to an embodiment, one or more motion information sets are derived from plural pieces of motion information stored in a computer readable storage medium. Each of the one or more information sets includes two pieces of motion information between which any one of constituent elements differ in value. A predicted signal of a target region in a picture is generated by motion compensation, using two pieces of motion information in a motion information set selected from the one or more motion information sets. The two pieces of motion information in the selected motion information set are stored in the computer readable storage medium, and are used for generation of a predicted signal of another region.

This application is a continuation of PCT/JP2011/072449, filed Sep. 29,2011, which claims the benefit of the filing date pursuant to 35 U.S.C.§119(e) of JP2010-226472, filed Oct. 6, 2010, both of which areincorporated herein by reference.

TECHNICAL FIELD

Embodiments of the present invention relate to an image predictiveencoding device, an image predictive encoding method, an imagepredictive encoding program, an image predictive decoding device, animage predictive decoding method, and an image predictive decodingprogram and, more particularly, to an image predictive encoding device,an image predictive encoding method, an image predictive encodingprogram, an image predictive decoding device, an image predictivedecoding method, and an image predictive decoding program to generate apredicted signal of a target block, using motion information ofneighboring blocks.

BACKGROUND ART

The compression encoding technologies are used for efficienttransmission and storage of still pictures and video data. Thetechniques defined in MPEG-1 to 4 and ITU (InternationalTelecommunication Union) H.261 to H.264 are commonly used for videodata.

SUMMARY

Bi-predictive prediction involves averaging of two similar predictedsignals, whereby noise in the predicted signals can be more effectivelyremoved by a smoothing effect. For this purpose, it is effective toacquire two signals with slightly different motion vectors from anidentical reference picture. In bi-predictive prediction using themotion information of neighboring blocks, the selection of motioninformation to be used in generation of two predicted signals can berestricted by reference picture lists. Because of this restriction, evenif a set of two pieces of motion information of neighboring blocksinclude motion vectors of close values based on the same referencepicture, they cannot be selected.

In the example of FIG. 7, an illustrated block 400 is an encoding targetblock and blocks 401 to 403 are neighboring blocks to the target block,each of the three neighboring blocks may have two pieces of motioninformation identified by List0 and List1. The two predicted signals canbe generated by selecting one of the three pieces of motion informationidentified by List0 and one of the three pieces of motion informationidentified by List1, which are the motion information of the threeneighboring blocks. In general, a bit count for reference index ref_idxcan become smaller when the reference index ref_idx is “0”; therefore,it can be the case that all the reference indices ref_idx in the motioninformation of neighboring blocks are 0. In the case where the referenceindex is “0,” use of the reference picture lists (521) in (A) of theexample of FIG. 6 can result in acquiring two predicted signals from areference picture with the frame number (frame_num) of “3” and from areference picture with the frame number of “2,” respectively. In thiscase, it is not feasible to achieve a high smoothing effect.

As another example, two reference picture lists can be composed ofdifferent reference pictures. In the case, as shown in (B) of theexample of FIG. 5, where a picture 510 identified by the frame number of“3” is an encoding target picture, where pictures 507, 508, 509, and 511identified by the frame numbers of “0,” “1,” “2,” and “4”, respectively,are reconstructed pictures, and where the reference picture lists arelists 522 shown in (B) of the example of FIG. 6, the two predictedsignals can be generated from different reference pictures. In thiscase, it is also not feasible to achieve a high smoothing effect.

An aspect of the image predictive encoding and decoding system relatesto image predictive encoding.

The image predictive encoding and decoding system may include an imagepredictive encoding device. The image predictive encoding device caninclude a region partition means which partitions an input picture intoa plurality of regions; predicted signal generation means whichdetermines motion information for acquisition of a pixel signal having ahigh correlation with a pixel signal of a target region of an encodingtarget out of the plurality of regions partitioned by the regionpartition means, from a reconstructed picture, and which generates apredicted signal of the target region from the reconstructed picture onthe basis of the motion information; motion information recording meanswhich stores the motion information; residual signal generation meanswhich generates a residual signal of a difference between the predictedsignal of the target region and the pixel signal of the target region;residual signal encoding means which encodes the residual signalgenerated by the residual signal generation means; residual signalreconstruction means which decodes encoded date generated by theresidual signal encoding means, to generate a reconstructed residualsignal; addition means which adds the predicted signal to thereconstructed residual signal to generate a reconstructed pixel signalof the target region; and picture recording means which stores thereconstructed pixel signal generated by the addition means, as a part ofa reconstructed picture. The predicted signal generation meanscomprises: motion information derivation means which derives one or moremotion information sets from plural pieces of motion information storedin the motion information recording means, each of the one or moremotion information sets consisting of two pieces of motion informationbetween which any one of constituent elements differs in value. Theimage predictive encoding device can also include motion predictioninformation selection means which selects a motion information set to beused in generation of the predicted signal of the target region, fromthe one or more motion information sets and which directs storage of twopieces of motion information included in the selected set in the motioninformation recording means; and motion compensation means whichcombines two signals acquired from the reconstructed picture based onthe two pieces of motion information in the selected set, to generatethe predicted signal of the target region.

The image predictive encoding and decoding system may perform an imagepredictive encoding method comprising: partitioning an input pictureinto a plurality of regions; generating a predicted signal of a targetregion of an encoding target out of the plurality of regions, saidgenerating including determining motion information for acquisition of apixel signal having a high correlation with a pixel signal of the targetregion from a reconstructed picture and generating the predicted signalfrom the reconstructed picture on the basis of the motion information;storing the motion information with motion information recording means;generating a residual signal of a difference between the predictedsignal of the target region and the pixel signal of the target region;encoding the residual signal; decoding encoded data generated duringsaid encoding of the residual signal, to generate a reconstructedresidual signal; adding the predicted signal to the reconstructedresidual signal to generate a reconstructed pixel signal of the targetregion; and storing the reconstructed pixel signal as a part of areconstructed picture, by picture recording means. In addition, the stepof generating the predicted signal comprises: deriving one or moremotion information sets out of plural pieces of motion informationstored in the motion information recording means, each consisting of twopieces of motion information between which any one of constituentelements differs in value; selecting, from the one or more motioninformation sets, a motion information set to be used in generation ofthe predicted signal of the target region, and directing storage of twopieces of motion information in the selected set in the motioninformation recording means; and combining two signals acquired from thereconstructed picture on the basis of the two pieces of motioninformation in the selected set, to generate the predicted signal of thetarget region.

The image predictive encoding and decoding system may also include animage predictive encoding program that makes a computer function as theregion partition means, the predicted signal generation means, themotion information recording means, the residual signal generationmeans, the residual signal encoding means, the residual signalreconstruction means, the addition means, and the picture recordingmeans. The predicted signal generation means comprises the motioninformation derivation means, the prediction information selectionmeans, and the motion compensation means.

Image predictive encoding with the image predictive encoding anddecoding system permits two pieces of motion information between whichany constituent element differs in value to be selected without beingrestricted by reference picture lists. As a result, the effect ofsmoothing can be achieved in the generation of the predicted signal.

In an embodiment, the image predictive encoding of the image predictiveencoding and decoding system may be configured as follows: the picturerecording means can store a plurality of reconstructed pictures, eachpiece of the motion information contains information to identify thereconstructed picture, and a motion vector, the reconstructed picturesstored in the picture recording means are identified using respectiveframe numbers in encoding of the reconstructed pictures, and a derivedmotion information set is a motion information set including two piecesof motion information having an identical frame number of thereconstructed picture and different values of motion vectors.

In an embodiment, the image predictive encoding of the image predictiveencoding and decoding system may be configured as follows: the picturerecording means can store a plurality of reconstructed pictures, theplurality of reconstructed pictures stored in the picture recordingmeans are identified using two reference picture lists each consistingof one or more reconstructed pictures, each piece of the motioninformation contains information to identify the two reference picturelists, and the motion vector, and a derived motion information set is amotion information set including two pieces of motion information havingidentical information to identify the reference picture lists, anddifferent values of motion vectors.

In an embodiment, the image predictive encoding of the image predictiveencoding and decoding system may be configured as follows: the motioninformation set to be used in generation of the predicted signal of thetarget region can be selected from plural pieces of motion informationassociated with a region neighboring the target region.

In an embodiment, the image predictive encoding of the image predictiveencoding and decoding system may be configured as follows: informationto indicate the selected motion information set consisting of the twopieces of motion information can be encoded.

Another aspect of the image predictive encoding and decoding systemrelates to image predictive decoding.

The image predictive encoding and decoding system can include an imagepredictive decoding device. The image predictive decoding device caninclude: data analysis means which extracts encoded data of motioninformation for generation of a predicted signal of a target region of adecoding target out of a plurality of regions in a picture, and encodeddata of a residual signal from compressed data; motion informationrecording means which stores the motion information; motion compensationmeans which generates the predicted signal of the target region on thebasis of the motion information; residual signal decoding means whichdecodes a residual signal of the target region from the encoded data ofthe residual signal; picture recording means which adds the predictedsignal to the decoded residual signal to restore a pixel signal of thetarget region, and which stores the restored pixel signal as a part of areconstructed picture; motion information derivation means whichderives, from plural pieces of motion information stored in the motioninformation recording means, one or more motion information sets eachconsisting of two pieces of motion information between which any one ofconstituent elements differs in value; and motion informationdetermination means which determines a motion information set to be usedin generation of the predicted signal of the target region, out of theone or more motion information sets derived, and which directs storageof two pieces of motion information in the determined set in the motioninformation recording means, wherein the motion compensation meanscombines two signals acquired from the reconstructed picture on thebasis of the two pieces of motion information in the determined set, togenerate the predicted signal of the target region.

The image predictive encoding and decoding system can also perform animage predictive decoding method that includes: extracting encoded dataof motion information for generation of a predicted signal of a targetregion of a decoding target out of a plurality of regions in a picture,and encoded data of a residual signal from compressed data; storing themotion information with motion information recording means; generatingthe predicted signal of the target region on the basis of the motioninformation; restoring a decoded residual signal of the target regionfrom encoded data of the residual signal; adding the predicted signal tothe decoded residual signal to restore a pixel signal of the targetregion, and storing the restored pixel signal as a part of areconstructed picture with picture recording means; deriving one or moremotion information sets out of plural pieces of motion informationstored in the motion information recording means, each of the one ormore motion information sets consisting of two pieces of motioninformation between which any one of constituent elements differ invalue; and determining a motion information set to be used in generationof the predicted signal of the target region from among the derivedmotion information sets, and directing storage of two pieces of motioninformation in the determined set in the motion information recordingmeans, wherein the step of generating the predicted signal comprisescombining two signals acquired from the reconstructed picture on thebasis of the two pieces of motion information in the determined set, togenerate the predicted signal of the target region.

The image predictive encoding and decoding system can also include animage predictive decoding program that makes a computer function as thedata analysis means, the motion information recording means, the motioncompensation means, the residual signal decoding unit, the picturerecording means, the motion information derivation means, and the motioninformation determination means.

The image predictive encoding and decoding system can perform imagepredictive decoding that allows a picture to be restored from compresseddata generated by the image predictive encoding of the aforementionedaspect. Noise can be reduced in this restored picture.

In an embodiment, the image predictive decoding performed by the imagepredictive encoding and decoding system may include extracting encodeddata of indication information, the encoded data extracted out of pluralpieces of motion information stored in the motion information recordingmeans to identify one motion information set consisting of two pieces ofmotion information between which any one of constituent elements differin value, making prediction information decoding means decode theindication information, and extracting a motion information set to beused in generation of the predicted signal of the target region, out ofthe one or more motion information sets derived by the motioninformation derivation means, based on the decoded indicationinformation.

In an embodiment, the image predictive decoding by the image predictiveencoding and decoding system may be configured as follows: the picturerecording means can store a plurality of reconstructed pictures, themotion information contains information to identify the plurality ofreconstructed pictures, and motion vectors, the plurality ofreconstructed pictures stored in the picture recording means areidentified using respective frame numbers in decoding of the respectivereconstructed pictures, and a derived motion information set is a motioninformation set including two pieces of motion information having anidentical frame number of the reconstructed picture and different valuesof motion vectors.

In an embodiment, the image predictive decoding of the image predictiveencoding and decoding system may be configured as follows: the picturerecording means can store a plurality of reconstructed pictures, theplurality of reconstructed pictures stored in the picture recordingmeans are identified using two reference picture lists each consistingof one or more reconstructed pictures, each piece of the motioninformation is configured containing information to identify the tworeference picture lists, and the motion vector, and a derived motioninformation set is a motion information set including two pieces ofmotion information having identical information to identify thereference picture lists, and different values of motion vectors.

In an embodiment, the image predictive decoding of the image predictiveencoding and decoding system may be configured as follows: the motioninformation set to be used in generation of the predicted signal of thetarget region can be derived from plural pieces of motion informationassociated with a region neighboring the target region.

As described above, the system can include the image predictive encodingdevice, image predictive encoding method, image predictive encodingprogram, image predictive decoding device, image predictive decodingmethod, and image predictive decoding program capable of suppressingnoise of the predicted signal.

Other systems, methods, features and advantages will be, or will become,apparent to one with skill in the art upon examination of the followingfigures and detailed description. It is intended that all suchadditional systems, methods, features and advantages be included withinthis description, be within the scope of the invention, and be protectedby the following claims.

BRIEF DESCRIPTION OF DRAWINGS

The system, may be better understood with reference to the followingdrawings and description. The components in the figures are notnecessarily to scale, emphasis instead being placed upon illustratingthe principles of the system. Moreover, in the figures, like referencednumerals designate corresponding parts throughout the different views.

FIG. 1 illustrates an example configuration of an image predictiveencoding device according to an embodiment.

FIG. 2 illustrates an example configuration of a predicted signalgeneration unit shown in FIG. 1.

FIG. 3 is a flowchart showing an example of a process of a motioninformation derivation unit shown in FIG. 2.

FIG. 4 is a schematic diagram for explaining an example of a techniquefor selecting one piece of candidate prediction information fromcandidate motion prediction information (combination of two pieces ofmotion information) using neighboring pixels to a target block.

FIG. 5 is a schematic diagram for explaining examples of encoding ordersof pictures.

FIG. 6 is a drawing for explaining examples of reference picture lists.

FIG. 7 is a drawing for explaining examples of neighboring blocks.

FIG. 8 is a drawing for explaining other examples of neighboring blocks.

FIG. 9 is a flowchart showing an example procedure of an imagepredictive encoding method according to an embodiment.

FIG. 10 illustrates an example configuration of an image predictivedecoding device according to an embodiment.

FIG. 11 is a flowchart showing an example procedure of an imagepredictive decoding method according to an embodiment.

FIG. 12 illustrates an example of modules that are executable to performimage predictive encoding according to an embodiment.

FIG. 13 illustrates an example of modules that are executable to performimage predictive decoding according to an embodiment.

FIG. 14 illustrates an example hardware configuration of a computer forexecuting at least a portion of the image predictive encoding anddecoding system.

FIG. 15 is a perspective view of an example computer for executing theimage predictive encoding and decoding system.

FIG. 16 is a schematic drawing for explaining an example of a motionestimation process in inter-picture prediction.

FIG. 17 is a schematic diagram for explaining an example ofintra-picture prediction.

DESCRIPTION OF EMBODIMENTS

Some embodiments will be described below in detail with reference to thedrawings. In the drawings identical or equivalent portions will bedenoted by the same reference signs, without redundant description. Inthe present specification, “frames,” “pictures,” and “images” (such as501 to 511 in FIG. 5) mean the same thing.

In some example encoding techniques, a picture as an encoding target ispartitioned into a plurality of blocks and then an encoding process anda decoding process are carried out on a block basis. In intra-picturepredictive encoding, a predicted signal can be generated using apreviously-reproduced picture signal (or a restored signal of compressedpicture data) of a neighboring region present in the same picture as atarget block and then a difference signal of a difference between thepredicted signal and a pixel signal of the target block can be encoded.In inter-picture predictive encoding, a predicted signal can begenerated with compensation for motion with reference to apreviously-reproduced picture signal of a neighboring region present ina picture different from a target block, and a difference signal of adifference between the predicted signal and a signal of the target blockcan be encoded.

Intra-picture predictive encoding, such as using H.264, can includegenerating the predicted signal by extrapolating previously-reproducedpixel values of a region adjacent to a block serving as an encodingtarget, in predetermined directions. FIG. 17 is a schematic diagramexample for explaining an intra-picture prediction method, such as themethod used in ITU H.264. In (A) of FIG. 17, a target block 802 is ablock serving as an encoding target, and a pixel group 801 consisting ofpixels A to M adjacent to a boundary of the target block 802 is aneighboring region, which is a picture signal previously reproduced inpast processing.

In the prediction shown in (A) of FIG. 17, the predicted signal isgenerated by downwardly extending the pixel group 801 as neighboringpixels located immediately above the target block 802. In the predictionshown in (B) of FIG. 17, the predicted signal is generated byrightwardly extending previously-reproduced pixels (I to L) located onthe left side of the target block 804. A difference can be calculatedbetween each of nine predicted signals generated by the methods shown in(A) to (I) of FIG. 17 in the above-described manner, and the pixelsignal of the target block. These obtained differences can be comparedto select a candidate to provide a minimum difference, as an optimumpredicted signal. As described above, the predicted signal can begenerated by extrapolation of pixels.

In an example of inter-picture predictive encoding, the predicted signalcan be generated by searching previously-reproduced pictures for asignal similar to the pixel signal of the target block. Then encodingcan be performed of a motion vector as a spatial displacement amountbetween the target block and a region composed of the detected signal,and a residual signal of the difference between the pixel signal of thetarget block and the predicted signal. The searching technique for themotion vector for each block as described above may be described asblock matching.

FIG. 16 is a schematic diagram for explaining an example of a blockmatching process. The below will describe a an example procedure forgenerating a predicted signal for an example of a target block 702 on anencoding target picture 701. A picture 703 is a previously-reproducedpicture and a region 704 in the previously-reproduced picture is aregion located spatially at the same position as the target block 702.In the block matching, a search range 705 that includes the region 704is set. A region 706 with a signal to minimize the sum of absolutedifferences from the pixel signal of the target block 702 is detectedfrom this search range. The signal of this region 706 is determined tobe a predicted signal, and a displacement amount from the region 704 tothe region 706 is detected as a motion vector 707. An example processincludes preparing a plurality of reference pictures, selecting areference picture to be applied to block matching for each target block,and detecting reference picture selection information. In some cases,such as in H.264, a plurality of prediction types of different blocksizes are prepared for encoding of motion vector, in order to adapt forlocal feature changes of pictures.

In compression encoding of video data, an encoding order of pictures(frames or fields) may be optional. A number of different types oftechniques, such as three types of techniques, can be used regarding theencoding order in inter-picture prediction to generate the predictedsignal with reference to previously-reproduced pictures. A firsttechnique may use forward prediction to generate the predicted signalwith reference to a past previously-reproduced picture in a displayorder, a second technique is backward prediction to generate thepredicted signal with reference to a future previously-reproducedpicture in the display order, and a third technique is bidirectionalprediction to perform both forward prediction and backward predictionand average two predicted signals.

In some encoding examples, such as in H.264, the third technique can beperformed using reference picture lists, such as two reference pictureslists. Each of the reference picture lists may consist of a plurality ofpreviously-reproduced pictures as candidates for a reference picture. Inan example, the block matching may be carried out using the plurality ofreference pictures registered in each reference picture list to detecttwo regions corresponding to the region 706, and two predicted signalsthus detected can be averaged.

An example of reference picture lists will be described with FIGS. 5 and6. In (A) of FIG. 5, a picture 505 indicates an encoding target pictureand pictures 501 to 504 indicate reconstructed pictures. Each image(picture) is identified by a frame number (frame_num). Reference picturelist identification List0 and List1 in (A) of FIG. 6 indicate tworeference picture lists and in this example, four reference pictures areregistered in each of the two reference picture lists. Each referencepicture can be identified by a reference index (ref_idx).

Reconstructed pictures that can be registered in the reference picturelists are basically optional. As in the example table 521 shown in (A)of FIG. 6, all the reference pictures registered in the two referencepicture lists may be past reconstructed pictures. Since the twopredicted signals both are forward predicted signals in this case, theprediction technique to average the two predicted signals may be calledbi-predictive prediction, instead of the bidirectional prediction.

In bi-predictive prediction, two combinations (motion information) ofmotion vectors and reference indices may be encoded. Therefore, in orderto decrease the bit count, a technique of performing the bi-predictiveprediction using the motion information identified by List0 and themotion information identified by List1, out of the motion information ofneighboring blocks, may be used.

FIG. 1 is a drawing showing an example configuration of an imagepredictive encoding device 100 according to an example embodiment of theimage predictive encoding and decoding system. The image predictiveencoding device 100 may be a computing device or computer, including forexample software, hardware, or a combination of hardware and software,as described later, capable of performing the described functionality.The image predictive encoding device 100 may be one or more separatesystems or devices included in the image predictive encoding anddecoding system, or may be combined with other systems or devices withinthe image predictive encoding and decoding system. In other examples,fewer or additional blocks may be used to illustrate the functionalityof the image predictive encoding device 100. The image predictiveencoding device 100 shown in FIG. 1 includes an input terminal 101, ablock partition unit 102, a predicted signal generation unit 103, aframe memory 104, a subtraction unit 105, a transform unit 106, aquantization unit 107, an inverse quantization unit 108, an inversetransform unit 109, an addition unit 110, a quantized transformcoefficient encoding unit 111, an output terminal 112, a predictioninformation memory 113, and a prediction information encoding unit 114.The term “unit” is defined to include one or more executable parts of animage predictive encoding device or an image predictive decoding device.As described herein, the units are defined to include software,hardware, or some combination thereof executable by a processor(described later). Software included in the units may includeinstructions stored in memory, or any other computer readable storagemedium, that are executable by the processor, or any other processor.Hardware included in the units may include various devices, components,circuits, gates, circuit boards, and the like that are executable,directed, and/or controlled for performance by the processor. In anembodiment, the transform unit 106 and the quantization unit 107function as residual signal encoding unit or means, the inversequantization unit 108 and the inverse transform unit 109 function asresidual signal reconstruction unit or means, and the predictioninformation memory 113 functions as motion information recording unit ormeans. It is noted that the prediction information memory 113 may beincorporated in the predicted signal generation unit 103. In anembodiment, the quantized transform coefficient encoding unit 111 mayalso function as a part of the residual signal encoding unit or meansand, in this case, an element that decodes data generated by thequantized transform coefficient encoding unit 111 and outputs thedecoded data to the inverse quantization unit 108 constitutes a part ofthe residual signal reconstruction unit. Also, the block division unit102 may function as a region partition means. Further, the frame memory104 and the prediction information memory 113 may be one or morecomputer readable storage medium, as described later.

The input terminal 101 is a terminal that implements input of a signalof a video sequence consisting of a plurality of pictures.

The block partition unit 102 partitions each picture of an encodingtarget represented by a signal input from the input terminal 101, into aplurality of regions. In an embodiment, an encoding target picture maybe partitioned into a plurality of blocks each consisting of 8×8 pixels.These regions or blocks are target regions or target blocks to which theprocessing described below is applied. The size and shape of blocks arenot limited to this example. Blocks of different sizes may be mixed ineach picture.

The predicted signal generation unit 103 detects motion informationnecessary for generation of a predicted signal of each prediction blockin a target block and generates the predicted signal. There are noparticular restrictions on a method of generation of the predictedsignal, but it may be the inter-picture prediction or the intra-pictureprediction as described in the background art (though the intra-pictureprediction is not illustrated). In an embodiment, the motion informationmay be detected by the block matching shown in FIG. 16. In the blockmatching, the predicted signal to minimize the sum of absolutedifferences from the original signal (pixel signal) of the target blockis detected from a reconstructed picture acquired via line L104.

In an embodiment, the motion information may contain a motion vector, anidentification of a reference picture list (List0 and List1 in FIG. 5),and a reference index (ref_idx in FIG. 5).

When each reference picture list is composed of a plurality of referencepictures as shown in FIG. 6, the reference index is detectedsimultaneously with the motion vector, in order to indicate whichreference picture is a target of the motion vector in the motioninformation. When bi-predictive prediction is used, a prediction type(forward/backward/two-way) is also determined at the same time. When theprediction type is the forward prediction, the motion information withthe reference picture list identification of List0 is detected; when theprediction type is the backward prediction, the motion information withthe reference picture list identification of List1 is detected. In thecase of the bi-predictive prediction, two pieces of motion informationidentified by List0 and List1 are detected. The reconstructed picturesregistered in the reference picture lists shown in FIG. 6 may beautomatically determined in accordance with a predetermined rule, orinformation to specify the reconstructed pictures registered in thereference picture lists may be explicitly encoded in frame units or insequence units. In an embodiment, frame numbers may be used as theinformation, as shown in FIGS. 5 and 6.

Referring back to FIG. 1, the motion information and the prediction typeare output via line L103 c and via line L103 b to the predictioninformation memory 113 and to the prediction information encoding unit114, respectively.

The prediction information memory 113 stores the input motioninformation and prediction type.

The prediction information encoding unit 114 entropy-encodes the motioninformation of the encoding target block and outputs encoded data vialine L114 to the output terminal 112. There are no particularrestrictions on a method of the entropy encoding, but it may be thearithmetic coding, variable-length coding, or the like.

The predicted signal generated by the predicted signal generation unit103 is output via line L103 a to the subtraction unit 105 and to theaddition unit 110.

The subtraction unit 105 subtracts the predicted signal of the targetblock input via line L103 a, from the pixel signal of the target blockto generate a residual signal. The subtraction unit 105 outputs theresidual signal via line L105 to the transform unit 106.

The transform unit 106 performs a discrete cosine transform of the inputresidual signal to generate transform coefficients. The quantizationunit 107 quantizes the transform coefficients generated by the transformunit 106, to generate quantized transform coefficients. The quantizedtransform coefficient encoding unit 111 entropy-encodes the quantizedtransform coefficients generated by the quantization unit 107. Theencoded data generated by the quantized transform coefficient encodingunit 111 is output via line L111 to the output terminal 112. There areno particular restrictions on a method of the entropy encoding in thequantized transform coefficient encoding unit 111, but it may bearithmetic coding, variable-length coding, or the like.

The output terminal 112 outputs the information received from theprediction information encoding unit 114 and the information from thequantized transform coefficient encoding unit 111, together to theoutside.

The inverse quantization unit 108 performs inverse quantization of thequantized transform coefficients generated by the quantization unit 107,to generate transform coefficients. The inverse transform unit 109applies an inverse discrete cosine transform to the transformcoefficients generated by the inverse quantization unit 108, to restorea residual signal. The addition unit 110 adds the restored residualsignal to the predicted signal input provided via line L103 a, togenerate a reconstructed pixel signal of the encoding target block andstores the reconstructed pixel signal into the frame memory 104. It isalso possible to adopt other transform processes replacing the processesof the transform unit 106 and the inverse transform unit 109. It is alsonoted that the transform unit 106 and the inverse transform unit 109 arenot always essential. In this manner, the reconstructed pixel signal ofthe encoding target block is stored as a part of a reference picture orreconstructed picture in the frame memory 104, in order to be used forgeneration of a predicted signal of a subsequent encoding target block.

In an embodiment, the predicted signal generation unit 103 can alsogenerate the predicted signal of the target block by making use of themotion information associated with blocks neighboring the target block(neighboring blocks). The motion information associated with aneighboring block is motion information used in generation of thepredicted signal when the neighboring block was an encoding target. Themotion information associated with the neighboring blocks is stored inthe prediction information memory 113. For each block, information toindicate that the predicted signal was generated by making use of themotion information associated with the neighboring blocks, orinformation to indicate that the predicted signal was generated bymaking use of the motion vector detected by the block matching may beincluded in an encoded state in compressed data.

Now, let us consider the case where the predicted signal of the targetblock is generated by the bi-predictive prediction. It will be describedbelow using an example of (A) in FIG. 7. Neighboring blocks are twoblocks 401 and 402 neighboring the target block 400, which are locatedimmediately on the left side of and immediately above the target block400, and the motion information associated with these blocks is used forthe prediction. In this case, if the prediction type associated with thetwo neighboring blocks is the bi-predictive prediction, up to fourpieces of motion information can be used for generation of the predictedsignal of the target block. The predicted signal generation unit 103selects two pieces of motion information having different values ofmotion vectors or different values of frame numbers of referencepictures, from the four pieces of motion information as candidates (fourpieces of candidate motion information). A frame number (frame_num) of areference picture can be identified by a combination of a referencepicture list identification (List0 or List1) and a reference index(ref_idex). Then two predicted signals are acquired from the framememory 104 on the basis of the respective pieces of motion information,and the two predicted signals are averaged in pixel units to obtain apredicted signal. If the two predicted signals resemble each other, theprediction performance is improved by the effect of noise removal owingto smoothing.

The prediction information encoding unit 114 encodes information(indication information) to identify the two pieces of motioninformation used in the generation of the predicted signal of the targetblock out of the four pieces of candidate motion information, instead ofthe motion information. Therefore, the predicted signal may be generatedin a bit count that is smaller than that of the encoded data obtained byencoding the motion information. The indication information may be twopieces of indication information to identify the respective pieces ofcandidate motion information, or may be a piece of indicationinformation to identify the combination of the two pieces of candidatemotion information.

The prediction information memory 113 stores the two pieces of motioninformation actually used in the generation of the predicted signal ofthe target block, as motion information associated with the targetblock. The two pieces of motion information may be stored as they are,but since the existing image predictive encoding and decoding processeshave the restriction to refer to the two reference picture lists ofList0 and List1, the two pieces of motion information can be stored asdescribed below.

Namely, when the two pieces of motion information have the samereference picture list identification, the reference picture listidentification in the two pieces of motion information can be modifiedin accordance with a predetermined rule. For example, of the two piecesof motion information, the reference picture list identification of onepiece of motion information with a smaller reference index (ref_idx) isassigned to List0, and the reference picture list identification of theother piece of motion information is assigned to List1.

When the two pieces of motion information also have the same referenceindex, the reference picture list identification of one piece of motioninformation with the motion vector having the smaller sum of absolutevalues of respective components thereof is assigned to List0, and thereference picture list identification of the other piece of motioninformation is assigned to List1.

If this assignment results in referring to a frame number different fromthe frame number before the assignment, the reference index (ref_idx)can be modified so as to enable reference to the frame number before theassignment.

If the two reference picture lists do not include any identical framenumber as shown in (B) of FIG. 6, the motion information may be managedusing reference picture lists (523) extended so as to include the sameframe numbers as shown in (C) of FIG. 6. The extended reference picturelists may be used directly in the case where the predicted signal of thetarget block is generated using the motion information associated withneighboring blocks. Since the frame numbers are recorded, they are alsoapplicable to cases where the motion vector is scaled according to thedistance in the temporal direction between pictures (e.g., each vectorcomponent is doubled when the motion vector to the target of the picture504 in (A) of FIG. 5 is converted into the motion vector to the targetof the picture 503). Since the motion vector storage methods describedherein can be converted from one to another, any one of the storagemethods can be applied with no influence on the encoding result.

The processing of the predicted signal generation unit 103 will befurther described in detail. FIG. 2 is a drawing showing a configurationof the predicted signal generation unit 103 according to an embodiment.As shown in FIG. 2, the predicted signal generation unit 103 is providedwith a motion information derivation unit 121, a motion predictioninformation selection unit 122, and a motion compensation unit 123. Inthis drawing, the part of motion detection by the conventional blockmatching is omitted.

When both of the prediction types of the immediately above andimmediately left blocks used as neighboring blocks are bi-predictiveprediction, the motion information derivation unit 121 compares fourpieces of motion information associated with these neighboring blocks.The motion information derivation unit 121 defines motion informationhaving a different value of any one of elements constituting the motioninformation, which are the motion vector, the reference picture listidentification, and the reference index, from those of the other threepieces of motion information, as candidate motion information applicableto the prediction of the target block. If there are two or more piecesof motion information with the same motion vector value and the sameframe number value of the reference picture (the frame number of thereference picture can be identified by the combination of the referencepicture list and the reference index), one of them is defined ascandidate motion information. Then the motion information derivationunit 121 derives a set of any two pieces of candidate motioninformation, as candidate bi-predictive prediction information. In thiscase, up to six pieces of candidate bi-predictive prediction informationcan be derived. In cases where at least one of the immediately above andimmediately left blocks to the target block is employed as a neighboringblock and where the prediction is bi-predictive prediction, combinationpatterns of two pieces of motion information as many as the patternsdescribed below can be derived from the neighboring blocks. Namely, thecombination patterns are as follows.

(1) 2 patterns: in cases where two pieces of motion information of theimmediately above block (402) or the immediately left block (401) areemployed.

(2) 4 patterns: in cases where one piece of motion information of theimmediately above block (402) and one piece of motion information of theimmediately left block (401) are employed and where the referencepicture lists in the selected two pieces of motion information aredifferent.

(3) 2 patterns: in cases where one piece of motion information of theimmediately above block (402) and one piece of motion information of theimmediately left block (401) are employed and where the referencepicture lists in the selected two pieces of motion information are thesame.

FIG. 3 shows a flowchart of the processing carried out by the motioninformation derivation unit 121. First, in step S301 the number N ofneighboring blocks to be used in the prediction of the target block isset and the current neighboring block number n is set to “0.” Forexample, n=0 is set for block 401 and n=1 for block 402. Furthermore,the number L of pieces of candidate motion information is initialized to0. Then the motion information and prediction type of the n-th block areacquired from the prediction information memory 113 (step S302).Thereafter, the number M of pieces of motion information associated withthe n-th neighboring block is set based on the prediction type (stepS303).

Next, the m-th motion information of the n-th neighboring block iscompared with the L pieces of candidate motion information (step S304).If all the pieces of candidate motion information are different from themotion information of the comparison target, or if L=0, the processingproceeds to step S305. In step S305, the m-th motion information of then-th neighboring block is added to the candidate motion information andthe value of L is given an increment of 1. Thereafter, the processingproceeds to step S306.

If step S304 results in finding the same as the m-th motion informationin the candidate motion information, the processing proceeds to stepS306. In step S304, if values of two motion vectors in comparison areidentical and if frame numbers corresponding to two sets of referenceindices and reference picture list identifications in comparison areidentical, the motion information is determined to be identical evenwith difference of the reference picture lists. A reason for it is thatwhen the motion vectors of two pieces of motion information and theframe numbers of reference pictures thereof are identical, predictedsignals generated from these pieces of motion information becomecoincident.

In step S306, the value of m is given an increment of 1. In subsequentstep S307, it is determined whether the processes of steps S304 to S306are completed, for all the pieces of motion information associated withthe n-th neighboring block; if they are completed, the processingproceeds to step S308; if they are not completed, the processing returnsto step S304.

In step S308, the value of the number n of the neighboring block isgiven an increment of 1. Then in step S309, it is determined whether theprocessing is completed for all the neighboring blocks. If it iscompleted, the processing proceeds to step S310; if it is not completed,the processing returns to step S302.

Finally, in step S310, every set of two pieces of motion informationhaving different values of motion vectors or different values of framenumbers of reference pictures (the frame number of each referencepicture can be identified by the combination of the reference picturelist and the reference index) is derived from the L pieces of candidatemotion information and is set as candidate bi-predictive predictioninformation.

Concerning the motion information from which neighboring blocks themotion information derivation unit 121 uses, it may be preliminarilydetermined for both of the encoding side and the decoding side.Furthermore, information to specify the motion information from whichneighboring blocks the motion information derivation unit 121 used maybe transmitted in an encoded state to the decoding side.

The motion prediction information selection unit 122 selects an optimumpiece from the candidate bi-predictive prediction information input vialine L121 from the motion information derivation unit 121. Specifically,it selects the candidate bi-predictive prediction information tominimize the sum of absolute differences between the “signal obtained asan average of two signals acquired via line L104 from the frame memory104 on the basis of two pieces of candidate motion information” and the“pixel signal of the target block acquired via line L102.” Thenindication information to identify the selected candidate bi-predictiveprediction information or to identify the two pieces of candidate motioninformation constituting the selected candidate bi-predictive predictioninformation is output via line L103 b to the prediction informationencoding unit.

Furthermore, the motion prediction information selection unit 122outputs the selected two pieces of motion information via line L103 c tothe prediction information memory 113 so as to make the informationstored therein, and outputs the information via line L122 to the motioncompensation unit 123. The motion compensation unit 123 averages tworeconstructed pixel signals extracted from the reconstructed picturesacquired via line L104, based on the two input pieces of motioninformation, to generate a predicted signal, and outputs the predictedsignal via line L103 a to the subtraction unit 105 and to the additionunit 110.

There are no particular restrictions on the technique of selecting oneoptimum piece of candidate bi-predictive prediction information in themotion prediction information selection unit 122. For example, as shownin FIG. 4, instead of a pixel signal of a target block 461 in anencoding target picture 451, a reconstructed region (462) of an invertedL-shape neighboring the block 461 may be used. Since the region 462 isthe reconstructed region, the decoding device can also use it. As shownin the drawing, a predicted region (472) corresponding to the region 462can be acquired from the reference picture 452 on the basis of one pieceof motion information 473. For every piece of candidate bi-predictiveprediction information, inverted L-shape regions can be acquired fromtwo pieces of candidate motion information and signals thereof areaveraged to find the candidate bi-predictive prediction information tominimize the sum of absolute differences from the region 462. Thistechnique allows the decoding device to select the same candidatebi-predictive prediction information as that selected by the encodingdevice, and thus provides the advantage of eliminating the need fortransmission of the indication information.

In this manner, the predicted signal with a small difference from theoriginal signal is generated from the motion information associated withthe blocks neighboring the target block and two pieces of motioninformation having different values of motion vectors or differentvalues of frame numbers of reference pictures (the frame number of eachreference picture can be identified by the combination of the referencepicture list and the reference index) are selected, whereby thebi-predictive prediction of the target block can be implemented withouttransmission of the motion information. Since the motion informationcontains the information to identify the reference picture list, itbecomes feasible to perform the bi-predictive prediction using themotion information with the same reference picture list. Therefore, evenif the reference pictures in two reference picture lists are completelydifferent, the bi-predictive prediction can be carried out using motionvectors of the same reference picture. For example, when thebi-predictive prediction with two similar signals is carried out usingthe motion vectors of the same reference picture, improvement inprediction performance based on the smoothing effect may be expected.

FIG. 9 is a flowchart showing a procedure of an image predictiveencoding method according to an embodiment. First, the block partitionunit 102 partitions an input picture into 8×8 encoding blocks (the inputpicture may be partitioned in any other block size or shape or blocks ofdifferent sizes may be mixed in a picture).

Next, the motion information derivation unit 121 acquires the motioninformation associated with blocks neighboring a target block of anencoding target, from the prediction information memory 113 and extractsmotion information having a different value of any one of theconstituent elements thereof from the other motion information, ascandidate motion information applicable to the prediction of the targetblock. Then a set of any two pieces of candidate motion information isderived as candidate bi-predictive prediction information (step S300).The details of this step S300 were already described with reference toFIG. 3.

Next, the motion prediction information selection unit 122 selects oneto be used in generation of the predicted signal of the target block,from plural pieces of candidate bi-predictive prediction information.Then the motion prediction information selection unit 122 determines theindication information to identify the selected piece of candidatebi-predictive prediction information or to identify the two pieces ofcandidate motion information constituting the selected piece ofcandidate bi-predictive prediction information (step S320). Insubsequent step S101, the selected two pieces of motion information arestored into the prediction information memory 113 and the predictioninformation encoding unit 114 encodes the indication information.

Next, in step S102 the motion compensation unit 123 acquires pixelsignals from two reconstructed pictures acquired from the frame memory104, based on the selected motion information, and averages these pixelsignals to generate a predicted signal of the target block. In theprediction technique not using the motion information of neighboringblocks, the motion information is detected by the block matchingdescribed in FIG. 16 and the predicted signal of the target block isgenerated based on the motion information. The prediction informationencoding unit 114 encodes the motion information and information forswitching of the two prediction techniques. The motion information isstored into the prediction information memory 113.

Next, the residual signal indicative of the difference between the pixelsignal of the encoding target block and the predicted signal istransformed and encoded by the transform unit 106, the quantization unit107, and the quantized transform coefficient encoding unit 111 (stepS103). Encoded data of the indication information and quantizedtransform coefficients is output through the output terminal 112 (stepS104).

After these processes or in parallel with these processes, the encodedresidual signal is decoded by the inverse quantization unit 108 and theinverse transform unit 109, for predictive encoding of a subsequentencoding target block. Then the addition unit 110 adds the decodedresidual signal to the predicted signal to reconstruct a signal of theencoding target block. The reconstructed signal is stored as a referencepicture (reconstructed picture signal) into the frame memory 104 (stepS105). If the processing is not completed for all the encoding targetblocks, the processing returns to step S101 to carry out the processingfor the next encoding target block. If the processing is completed forall the encoding target blocks, the processing is terminated (stepS106).

Next, an image predictive decoding device according to an embodiment ofthe image predictive encoding and decoding system will be described.FIG. 10 is a block diagram showing the image predictive decoding deviceaccording 200 to an embodiment of the image predictive encoding anddecoding system. The image predictive decoding device 200 may be acomputing device or computer, including for example software, hardware,or a combination of hardware and software, as described later, capableof performing the described functionality. The image predictive decodingdevice 200 may be one or more separate systems or devices included inthe image predictive encoding and decoding system, or may be combinedwith other systems or devices within the image predictive encoding anddecoding system. In other examples, fewer or additional blocks may beused to illustrate the functionality of the image predictive decodingdevice 200. The image predictive decoding device 200 shown in FIG. 10may include an input terminal 201, a data analysis unit 202, an inversequantization unit 203, an inverse transform unit 204, an addition unit205, an output terminal 206, a quantized transform coefficient decodingunit 207, a prediction information decoding unit 208, a frame memory104, a motion information determination unit 209, a motion informationderivation unit 121, a prediction information memory 113, and a motioncompensation unit 126. The inverse quantization unit 203 and the inversetransform unit 204 function as residual signal decoding unit or meansand the prediction information memory 113 functions as motioninformation recording unit or means. The residual signal decodingperformed by the inverse quantization unit 203 and the inverse transformunit 204 may be performed by any means other than these in otherembodiments. Furthermore, the inverse transform unit 204 may be excludedin other embodiments. Also, the frame memory 104 and the predictioninformation memory 113 may be one or more computer readable storagemedium, as described later.

The input terminal 201 implements input of compressed data compressivelyencoded by the aforementioned image predictive encoding method. Thiscompressed data contains, for each of the blocks, encoded data obtainedby entropy encoding of quantized transform coefficients generated bytransform and quantization of the residual signal, encoded data of theprediction information, i.e., encoded data of motion information forgeneration of the predicted signal of the block or indicationinformation to identify the motion information to be used in generationof the predicted signal of the decoding target block from the motioninformation associated with neighboring blocks, and encoded data ofinformation to indicate which of the motion information and theindication information is encoded.

The data analysis unit 202 analyzes the compressed data input into theinput terminal 201, separates the compressed data into encoded data ofquantized transform coefficients and encoded data of predictioninformation, for a decoding target block, and outputs these encoded datavia line L202 a and via line L202 b to the quantized transformcoefficient decoding unit 207 and to the prediction information decodingunit 208, respectively.

The prediction information decoding unit 208 entropy-decodes the encodeddata of motion information or indication information. The decoded motioninformation is sent via line L208 to the motion informationdetermination unit 209 and is then output without change via line L207 aand line L207 b to the motion compensation unit 126 and the predictioninformation memory 113. The prediction information memory 113 stores themotion information and the motion compensation unit 126 acquires a pixelsignal from a reconstructed picture acquired from the frame memory 104,based on the motion information, and generates the predicted signal ofeach prediction block in the decoding target block, using the pixelsignal. The generated predicted signal is output via line L126 to theaddition unit 205.

On the other hand, when the prediction information decoding unit 208entropy-decodes the encoded data of indication information, the motioninformation derivation unit 121 functions. The motion informationderivation unit 121, as described in detail with reference to FIG. 3,acquires the motion information associated with blocks neighboring thedecoding target block, from the prediction information memory 113, andextracts information having different values of any one of constituentelements thereof, as candidate motion information applicable to theprediction of the decoding target block. Then the motion informationderivation unit 121 derives a set of any two pieces of candidate motioninformation as candidate bi-predictive prediction information andoutputs the candidate bi-predictive prediction information via line L121to the motion information determination unit 209.

The motion information determination unit 209 selects one piece ofcandidate bi-predictive prediction information identified by theindication information input via line L208, from plural pieces ofcandidate bi-predictive prediction information input via line L121. Thetwo pieces of motion information constituting the selected candidatebi-predictive prediction information are output via line L207 a and vialine L207 b to the motion compensation unit 126 and to the predictioninformation memory 113. The prediction information memory 113 stores themotion information and the motion compensation unit 126 acquires pixelsignals from reconstructed pictures acquired from the frame memory 104,based on the two pieces of motion information, and generates a predictedsignal of each prediction block in the decoding target block, using thepixel signals. The generated predicted signal is output via L126 to theaddition unit 205.

The quantized transform coefficient decoding unit 207 entropy-decodesthe encoded data of quantized transform coefficients of the residualsignal in the encoding target block, and outputs the restored quantizedtransform coefficients via line L207 to the inverse quantization unit203.

The inverse quantization unit 203 performs inverse quantization of thequantized transform coefficients of the decoding target block input vialine L207, to generate transform coefficients. The inverse transformunit 204 performs an inverse discrete cosine transform of the transformcoefficients to restore a residual signal. The residual signal is outputto the addition unit 205 via line L204.

The addition unit 205 adds the predicted signal generated by thepredicted signal generation unit 103, to the residual signal restored bythe inverse quantization unit 203 and the inverse transform unit 204, togenerate a decoded pixel signal of the decoding target block. Thedecoded pixel signal is output via line L205 to the output terminal 206and the frame memory 104. The output terminal 206 outputs areconstructed picture including the decoded pixel signal, to the outside(e.g., a display).

The frame memory 104 stores as a reference picture the reconstructedpicture output from the addition unit 205, which is used as areconstructed picture for reference for the next decoding process.

An image predictive decoding method according to an embodiment will bedescribed below with reference to FIG. 11. First, compressed data isinput through the input terminal 201 (step S201). Then the data analysisunit 202 performs the data analysis of the compressed data and extractsthe encoded data of the motion information for generation of thepredicted signal of the block or the indication information to identifythe motion information used in the generation of the predicted signal ofthe decoding target block from the motion information associated withneighboring blocks, the encoded data of the information to indicatewhich of the motion information and the indication information isencoded, and the encoded data of quantized transform coefficients. Theencoded data of motion information or the encoded data of indicationinformation is decoded by the prediction information decoding unit 208.

When the motion information is restored, the motion compensation unit126 generates the predicted signal of the decoding target block, basedon the restored motion information. The restored motion information isstored in the prediction information memory 113.

When the indication information is restored, the motion informationderivation unit 121 acquires the motion information of neighboringblocks from the prediction information memory 113. The motioninformation derivation unit 121 may acquire the motion information frompredetermined neighboring blocks. The motion information may be acquiredfrom neighboring blocks, based on the information transmitted from theencoding side.

Next, the motion information derivation unit 121, as shown in detail inFIG. 3, extracts motion information between which any one of constituentelements differs in value out of the acquired pieces of motioninformation, as candidate motion information applicable to theprediction of the decoding target block. Then the motion informationderivation unit 121 derives a set of any two pieces of candidate motioninformation as candidate bi-predictive prediction information (stepS300).

Next, the motion information determination unit 209 selects one piece ofbi-predictive prediction information from the derived pieces ofcandidate bi-predictive prediction information, using the restoredindication information, to restore two pieces of motion information(step S203).

Thereafter, the motion compensation unit 126 generates the predictedsignal of the decoding target block, based on the restored two pieces ofmotion information, and stores the two pieces of motion information intothe prediction information memory (S207).

Next, the quantized transform coefficients decoded by the quantizedtransform coefficient decoding unit 207 are inversely quantized by theinverse quantization unit 203 and the inverse transform unit 204 furtherapplies the inverse transform to the output from the inversequantization unit 203 to generate a decoded residual signal (S208).

Next, the generated predicted signal is added to the decoded residualsignal to generate a decoded signal, and this decoded signal is storedinto the frame memory 104, for decoding of the next decoding targetblock (step S209). Thereafter, when the determination in step S210results in determining that there is the next compressed data, theprocesses of S202 to S209 are repeated (S210), thereby processing alldata to the last.

FIG. 12 is a drawing showing a configuration of a program for causing acomputer to execute image predictive encoding according to an embodimentof the image predictive encoding and decoding system. The imagepredictive encoding program P100 may be at least part of the previouslydiscussed respective modules, and may include a block partition moduleP101, a predicted signal generation module P103, a storage module P104,a subtraction module P105, a transform module P106, a quantizationmodule P107, an inverse quantization module P108, an inverse transformmodule P109, an addition module P110, a quantized transform coefficientencoding module P111, a prediction information storage module P113, anda prediction information encoding module P114. The predicted signalgeneration module P103 includes a motion information derivation moduleP121, a motion prediction information selection module P122, and amotion compensation module P123.

The functions that the block partition module P101, predicted signalgeneration module P103, storage module P104, subtraction module P105,transform module P106, quantization module P107, inverse quantizationmodule P108, inverse transform module P109, addition module P110,quantized transform coefficient encoding module P111, predictioninformation storage module P113, prediction information encoding moduleP114, motion information derivation module P121, motion predictioninformation selection module P122, and motion compensation module P123cause a computer to execute may be at least part of the respectivefunctions of the block partition unit 102, predicted signal generationunit 103, frame memory 104, subtraction unit 105, transform unit 106,quantization unit 107, inverse quantization unit 108, inverse transformunit 109, addition unit 110, quantized transform coefficient encodingunit 111, prediction information memory 113, prediction informationencoding unit 114, motion information derivation unit 121, motionprediction information selection unit 122, and motion compensation unit123.

FIG. 13 is a drawing showing a configuration of a program for causing acomputer to execute image predictive decoding according to an embodimentof the image predictive encoding and decoding system. The imagepredictive decoding program P200 may be at least part of the previouslydiscussed respective modules, and may include a data analysis moduleP201, a prediction information decoding module P202, a motioninformation derivation module P121, a motion information determinationmodule P203, a prediction information storage module P205, a motioncompensation module P126, a quantized transform coefficient decodingmodule P206, an inverse quantization module P207, an inverse transformmodule P208, an addition module P209, and a storage module P104.

The functions that the data analysis module P201, prediction informationdecoding module P202, motion information derivation module P121, motioninformation determination module P203, prediction information storagemodule P205, motion compensation module P126, quantized transformcoefficient decoding module P206, inverse quantization module P207,inverse transform module P208, addition module P209, and storage moduleP104 cause a computer to execute may be at least part of the respectivefunctions of the data analysis unit 202, prediction information decodingunit 208, motion information derivation unit 121, motion informationdetermination unit 209, prediction information memory 113, motioncompensation unit 126, quantized transform coefficient decoding unit207, inverse quantization unit 203, inverse transform unit 204, additionunit 205, and frame memory 104.

The image predictive encoding program P100 or the image predictivedecoding program P200 configured as described above is stored in arecording medium 10 and executed by a computer described below.Alternatively or in addition, at least a portion of the previouslydescribed image predictive encoding program P100 and the imagepredictive decoding program P200 may be stored in a working memory,and/or a memory, which are also non-transitory computer readablemediums.

FIG. 14 is a drawing showing a hardware configuration of an examplecomputer for executing a program, such as a program recorded in arecording medium and FIG. 15 is a perspective view of an examplecomputer for executing a program, such as a computer program stored in arecording medium. Equipment that executes a program recorded in arecording medium is not limited to computers, but may be a DVD player, aset-top box, a cell phone, or the like provided with a CPU andconfigured to perform processing and control based on software.

As shown in the example of FIG. 14, a computer 30 is provided with areading device 12 such as a floppy disk drive unit, a CD-ROM drive unit,or a DVD drive unit, a communication port such as a universal serial busport (USB), Bluetooth port, an infrared communication port, or any othertype of communication port that allows communication with an externaldevice, such as another computer or memory device. The computer 30 mayalso include a working memory 14 that may include a resident operatingsystem, a memory 16 that stores data, such as at least part of a programstored in a recording medium 10. In addition, the working memory 14and/or the memory 16 may include the frame memory 104 and the predictioninformation memory 113. The working memory 14 and memory 16 may be oneor more non-transitory computer readable storage medium, and can includea solid-state memory such as a memory card or other package that housesone or more non-volatile memories, such as read-only memories. Further,the computer readable medium can include a random access memory or othervolatile re-writable memory. Additionally or alternatively, thecomputer-readable medium can include a magneto-optical or opticalmedium, such as a disk or tapes or any other non-transitory informationstorage medium to capture carrier wave signals such as a signalcommunicated over a transmission medium. A digital file attachment to ane-mail, stored in a storage medium, or other self-contained informationarchive or set of archives may be considered a non-transitorydistribution medium that is a tangible computer readable storage medium.Accordingly, the embodiments are considered to include any one or moreof a computer-readable storage medium or a non-transitory distributionstorage medium and other equivalents and successor information storagemedia, in which data or instructions may be stored. In addition, thecomputer 30 may have a user interface that includes a monitor device 18such as a display, a mouse 20 and a keyboard 22 as input devices, atouch screen display, a microphone for receipt of voice commands, asensor, or any other mechanism or device that allows a user to interfacewith the computer 30. In addition, the computer 30 may include acommunication device 24 for transmission/reception of data and others,and a central processing unit (CPU) 26 or processor to control executionof the program. The processor 26 may be one or more one or more generalprocessors, digital signal processors, application specific integratedcircuits, field programmable gate arrays, digital circuits, analogcircuits, combinations thereof, and/or other now known or laterdeveloped devices for analyzing and processing data. In one example,when the recording medium 10 is put into the reading device 12, thecomputer 30 becomes accessible to the image predictive encoding programP100 or the image predictive decoding program P200 at least a portion ofwhich can be stored in the recording medium 10, through the readingdevice 12, and becomes able to operate as the image predictive encodingdevice 100 or the image predictive decoding device 200, based on theimage predictive encoding program P100 or the image predictive decodingprogram P200.

As shown in the example of FIG. 15, the image predictive encodingprogram and the image predictive decoding program may be those providedin the form of computer data signal 40 superimposed on a carrier wave,through a network. In this case, the computer 30 can save at least aportion of the image predictive encoding program or the image predictivedecoding program received by the communication device 24, into thememory 16 and execute the image predictive encoding program or the imagepredictive decoding program following storage in the non-transitorycomputer readable medium.

The various embodiments were described above, but it should be notedthat the image predictive encoding and decoding system can furtherinclude modifications as described below.

(Candidates for Motion Information)

In the above embodiment the motion information is composed of the motionvector, the reference picture list identification (List0 and List1 inFIG. 5), and the reference index (ref_idx in FIG. 5), but the motioninformation may be configured by replacing the reference picture listidentification and the reference index with a frame number. A reason forit is that the same result and effect as in the above-describedembodiment using the motion information associated with neighboringblocks can be achieved by use of the frame numbers.

(Restrictions on Candidate Two-way Motion Information)

In the above embodiment the motion information derivation unit used allthe combinations of candidate motion information as candidate two-waymotion information, but only sets of motion information satisfying theconditions below may be adopted as candidate two-way motion information.

1) Only the immediately above and immediately left blocks to the targetblock.

2) Two pieces of motion information to indicate the same referencepicture.

3) Two motion vectors which are not identical but have close values (thedifference of absolute values of which is smaller than a certain value).Namely, the motion vector (MVXa,MVYa) of the immediately above block andthe motion vector (MVXb,MVYb) of the immediately left block satisfy thefollowing two relations:

(|MVXa−MVXb|!=0)∥(|MVYa−MVYb|!=0); and

(|MVXa−MVXb|<=N)&&(|MVYa−MVYb|<=N)(where N is a small value).

If there is no candidate two-way motion information satisfying theconditions, the other prediction process may be used withouttransmission of the indication information.

The number of candidate two-way motion information may be designated inframe units or in block units and information to specify the designatednumber may be included in compressed data. Furthermore, information toindicate such application of the restrictions on the candidate two-waymotion information may be encoded in frame units.

(Identification of Indication Information)

In determining the identification of the indication information, apriority order of the immediately left block (401) and the immediatelyabove block (402) may be determined using the motion information of theabove right block (404 in FIG. 8) to the target block (400 in FIG. 8).

If the motion vector of the immediately above block (MVXa,MVYa), themotion vector of the immediately left block (MVXb,MVYb), and the motionvector of the above left block (MVXc,MVYc) satisfy the relation of|MVXa−MVXc|+|MVYa−MVYc|<|MVXb−MVXc|+|MVYb−MVYc|, the motion informationof the immediately left block is given an identification with a smallerbit count. If the relation is not met, the motion information of theimmediately above block is given an identification with a smaller bitcount. When a higher priority is put on the motion information of theneighboring block assumed to have a higher correlation with the targetblock in this manner, the bit count of the indication information can bereduced.

(Derivation and Selection of Candidate Motion Vector)

In the above embodiment the derivation and selection of candidate motionvector are carried out for the horizontal and vertical components of themotion vector together, but the derivation and selection of candidatemotion vector may be individually performed for the horizontal componentand the vertical component of the motion vector. In the above embodimenta single motion vector of a neighboring block was defined as a candidatemotion vector, but it is also possible to define as a candidate motionvector an average of two motion vectors or a motion vector scaledaccording to the distance in the temporal direction between pictures(e.g., each vector component is doubled when the motion vector to thetarget of the picture 504 in (A) of FIG. 5 is converted into the motionvector to the target of the picture 503).

(Prediction Type)

In the above embodiment the prediction type for both of two neighboringblocks is the bi-predictive prediction, but the image predictiveencoding and decoding system can also be used in cases where one of themis forward or backward prediction. In such cases, up to three pieces ofcandidate bi-predictive prediction information are derived. In thesecases, combination patterns of two pieces of motion information as manyas the patterns described below can be derived from neighboring blocks.Namely, the combination patterns are as follows:

(1) 1 pattern: in cases where two pieces of motion information of theimmediately above block (402) or the immediately left block (401) areadopted;

(2) 1 pattern: in cases where one piece of motion information of theimmediately above block (402) and one piece of motion information of theimmediately left block (401) are employed and where the referencepicture lists about the two pieces of motion information are different;

(3) 1 pattern: in cases where one piece of motion information of theimmediately above block (402) and one piece of motion information of theimmediately left block (401) are employed and where the referencepicture lists about the two pieces of motion information are identical.

Furthermore, both of the two neighboring blocks may be the forward orbackward prediction, or one of them may be the forward prediction andthe other the backward prediction.

(Number of Neighboring Blocks)

There are no particular restrictions on the number of neighboring blocksapplicable to the prediction of the target block. As in (B) of FIG. 7,three blocks 401 to 403 may be used as neighboring blocks. It is alsopossible to use motion information associated with a block of apreceding frame. If the neighboring blocks (401-405) are furtherpartitioned as shown in FIG. 8, the motion information associated withsuch smaller blocks may be included in the candidate motion information.It is also possible to use motion information of a block which is notadjacent to the target block.

(N-Predictive Prediction)

In the above embodiment the prediction method of the target blockperformed using the motion information associated with the neighboringblocks is the bi-predictive prediction, but the prediction method doesnot have to be limited to it, and may be uni-predictive prediction ortri-predictive prediction. In this case, step S310 in FIG. 3 may beconfigured to derive candidate N-predictive prediction information andselect one therefrom. N pieces of motion information may be stored inthat case, but in the case where N is three or more, the motioninformation to be stored may be limited to two, by the reference indicesor the values of the motion vectors.

(Transform Unit and Inverse Transform Unit)

The transform process of the residual signal may be carried out in afixed block size or the transform process may be carried out by furtherpartitioning the target region according to a partial region.

(Color Signal)

In the description of the above embodiment there is no particulardescription about color format, but the predicted signal generationprocess may be carried out individually from a luminance signal, for acolor signal or a color difference signal. The processing may also becarried out in synchronism with the processing of the luminance signal.

The image predictive encoding and decoding system has been describedabove in detail on the basis of the described embodiments. However, itshould be noted that the image predictive encoding and decoding systemis by no means limited to the above embodiments. The present inventioncan be modified in many ways without departing from the scope and spiritof the invention. It will be apparent to those of ordinary skill in theart that many more examples and implementations are possible within thescope of the system. Accordingly, the system is not to be restrictedexcept in light of the attached claims and their equivalents

REFERENCE SIGNS LIST

100 image predictive encoding device; 101 input terminal; 102 blockpartition unit; 103 predicted signal generation unit; 104 frame memory;105 subtraction unit; 106 transform unit; 107 quantization unit; 108inverse quantization unit; 109 inverse transform unit; 110 additionunit; 111 quantized transform coefficient encoding unit; 112 outputterminal; 113 prediction information memory; 114 prediction informationencoding unit; 121 motion information derivation unit; 122 motionprediction information selection unit; 123 motion compensation unit; 201input terminal; 202 data analysis unit; 203 inverse quantization unit;204 inverse transform unit; 205 addition unit; 206 output terminal; 207quantized transform coefficient decoding unit; 208 predictioninformation decoding unit; 209 motion information determination unit.

1. An image predictive encoding device comprising: a processor: a blockpartition unit executable by the processor to partition an input pictureinto a plurality of regions; a predicted signal generation unitexecutable by the processor to determine motion information foracquisition of a pixel signal having a high correlation with a pixelsignal of a target region of an encoding target, the target region beingfrom among the plurality of regions partitioned by the block partitionunit, from a reconstructed picture, and the predicted signal generationunit also executable by the processor to generate a predicted signal ofthe target region from the reconstructed picture based on the motioninformation; a computer readable storage medium configured to store themotion information; a residual signal generation unit executable by theprocessor to generate a residual signal, the residual signal being adifference between the predicted signal of the target region and thepixel signal of the target region; a residual signal encoding unitexecutable by the processor to encode the residual signal generated bythe residual signal generation unit; a residual signal reconstructionunit executable by the processor to encode date generated by theresidual signal encoding unit, to generate a reconstructed residualsignal; an addition unit executable by the processor to add thepredicted signal to the reconstructed residual signal to generate areconstructed pixel signal of the target region; and the computerreadable storage medium further configured to store the reconstructedpixel signal generated by the addition unit, as a part of areconstructed picture, wherein the predicted signal generation unitcomprises: a motion information derivation unit executable by theprocessor to derive one or more motion information sets from pluralpieces of motion information stored in the computer readable storagemedium, each of the one or more motion information sets consisting oftwo pieces of motion information between which any one of constituentelements differ in value; a motion prediction information selection unitexecutable by the processor to select, from the one or more motioninformation sets, a motion information set to be used in generation ofthe predicted signal of the target region, the motion predictioninformation selection unit further executable by the processor to directstorage in the computer readable storage medium of two pieces of motioninformation included in the selected set; and a motion compensation unitexecutable by the processor to combine two signals acquired from thereconstructed picture based on the two pieces of motion information inthe selected set, to generate the predicted signal of the target region.2. The image predictive encoding device according to claim 1, whereinthe computer readable storage medium is configured to store a pluralityof reconstructed pictures, wherein each piece of the motion informationcontains a motion vector and information to identify the reconstructedpicture, wherein the reconstructed pictures stored in the computerreadable storage medium are identified using respective frame numbersassigned during encoding of the reconstructed pictures, and wherein themotion information derivation unit is executable by the processor toderive a motion information set including two pieces of motioninformation which contain motion vectors whose frame number of thereconstructed picture is identical, but whose values are different. 3.The image predictive encoding device according to claim 1, wherein thecomputer readable storage medium is configured to store a plurality ofreconstructed pictures, wherein the plurality of reconstructed picturesstored in the computer readable storage medium are identified using tworeference picture lists each consisting of one or more reconstructedpictures, wherein each piece of the motion information containsinformation to identify the two reference picture lists, and a motionvector, and wherein the motion information derivation unit is executableby the processor to derive motion information sets that include twopieces of motion information having identical information to identifythe reference picture lists, and different values of motion vectors. 4.The image predictive encoding device of claim 1, wherein the motioninformation derivation unit is executable by the processor to derive themotion information set used in generation of the predicted signal of thetarget region from plural pieces of motion information associated with aregion neighboring the target region.
 5. The image predictive encodingdevice of claim 1, further comprising a prediction information encodingunit executable by the processor to encode information indicating themotion information set consists of the two pieces of motion informationselected by the predicted signal generation unit.
 6. An image predictivedecoding device comprising: a processor; a data analysis unit executableby the processor to extract encoded data of motion information, theencoded data of motion information extracted out of a plurality ofregions in a picture for generation of a predicted signal of a targetregion of a decoding target, and the data analysis unit is furtherexecutable by the processor to extract encoded data of a residual signalfrom compressed data; a computer readable storage medium configured tostore the motion information; a motion compensation unit executable bythe processor to generate the predicted signal of the target regionbased on the motion information; a residual signal decoding unitexecutable by the processor to decode a residual signal of the targetregion from the encoded data of the residual signal; an addition unitexecutable by the processor to add the predicted signal to the decodedresidual signal to restore a pixel signal of the target region, theprocessor configured to store the restored pixel signal as a part of areconstructed picture in the computer readable storage medium; a motioninformation derivation unit executable by the processor to derive, fromplural pieces of motion information stored in the computer readablestorage medium, one or more motion information sets each consisting oftwo pieces of motion information between which any one of constituentelements differ in value; and a motion information determination unitexecutable by the processor to determine, from among the one or morederived motion information sets, a motion information set used ingeneration of the predicted signal of the target region, and the motioninformation determination unit is further executable by the processor todirect storage of two pieces of motion information included in thedetermined set in the computer readable storage medium; wherein themotion compensation unit is further executable by the processor tocombine two signals acquired from the reconstructed picture to generatethe predicted signal of the target region, the two signals combinedbased on the two pieces of motion information in the determined set. 7.The image predictive decoding device according to claim 6, wherein thedata analysis unit is executable by the processor to extract, fromplural pieces of motion information stored in the computer readablestorage medium, encoded data comprising indication information thatidentifies at least one motion information set consisting of two piecesof motion information between which any one of constituent elementsdiffer in value; the image predictive decoding device further comprisinga prediction information decoding unit executable by the processor todecode the indication information; wherein the motion informationdetermination unit is executable by the processor to extract, based onthe decoded indication information, a motion information set to be usedin generation of the predicted signal of the target region, the motioninformation set extracted out of the one or more motion information setsderived by the motion information derivation unit.
 8. The imagepredictive decoding device according to claim 6, wherein the computerreadable storage medium is configured to store a plurality ofreconstructed pictures, wherein the motion information containsinformation to identify the plurality of reconstructed pictures, andmotion vectors, wherein the plurality of reconstructed pictures storedin the computer readable storage medium are identified using respectiveframe numbers during decoding of the respective reconstructed pictures,and wherein the motion information derivation unit is executable by theprocessor to derive a motion information set including two pieces ofmotion information which contain motion vectors whose frame number ofthe reconstructed picture is identical but whose values are different.9. The image predictive decoding device according to claim 6, whereinthe computer readable storage medium is configured to store a pluralityof reconstructed pictures, wherein the plurality of reconstructedpictures stored in the computer readable storage medium are identifiedusing two reference picture lists, each consisting of one or morereconstructed pictures, wherein each piece of the motion information isconfigured to contain a motion vector, and information identifying thetwo reference picture lists, and wherein the motion informationderivation unit is executable by the processor to derive a motioninformation set including two pieces of motion information havingidentical information to identify the reference picture lists, anddifferent values of motion vectors.
 10. The image predictive decodingdevice according to claim 6, wherein the motion information derivationunit is executable by the processor to derive the motion information setused in generation of the predicted signal of the target region, fromplural pieces of motion information associated with a region neighboringthe target region.
 11. An image predictive encoding method comprising:partitioning an input picture into a plurality of regions with aprocessor; generating, with the processor, a predicted signal of atarget region of an encoding target out of the plurality of regions,said generating by the processor including: determining motioninformation for acquisition of a pixel signal from a reconstructedpicture having a high correlation with a pixel signal of the targetregion, and generating the predicted signal from the reconstructedpicture with the processor based on the motion information; storing themotion information in a computer readable storage medium; generating aresidual signal with the processor, the residual signal being adifference between the predicted signal of the target region and thepixel signal of the target region; the processor encoding the residualsignal; decoding with the processor encoded data generated by saidencoding of the residual signal, to generate a reconstructed residualsignal; the processor adding the predicted signal to the reconstructedresidual signal to generate a reconstructed pixel signal of the targetregion; and storing the reconstructed pixel signal as a part of areconstructed picture in the computer readable storage medium, whereinthe processor generating the predicted signal comprises: deriving, withthe processor, one or more motion information sets out of plural piecesof motion information stored in the computer readable storage medium,each of the one or more sets consisting of two pieces of motioninformation between which any one of constituent elements differ invalue; the processor selecting, from the one or more motion informationsets, a motion information set to be used in generation of the predictedsignal of the target region, and storing two pieces of motioninformation in the selected set in the computer readable storage medium;and the processor combining two signals acquired from the reconstructedpicture based on the two pieces of motion information in the selectedset, the two signals combined to generate the predicted signal of thetarget region.
 12. A non-transitory computer readable storage mediumcomprising instructions executed by a computer for image predictiveencoding, the computer readable storage medium comprising: instructionsexecutable by the computer to partition an input picture into aplurality of regions; instructions executable by the computer todetermine motion information for acquisition of a pixel signal from areconstructed picture, the pixel signal from the reconstructed picturehaving a high correlation with a pixel signal of a target region of anencoding target included in the plurality of regions; instructionsexecutable by the computer to generate a predicted signal of the targetregion from the reconstructed picture based on the motion information;instructions executable by the computer to store the motion information;instructions executable by the computer to generate a residual signal,the residual signal being a difference between the predicted signal ofthe target region and the pixel signal of the target region;instructions executable by the computer to encode the residual signal;instructions executable by the computer to decode the encoded residualsignal to generate a reconstructed residual signal; instructionsexecutable by the computer to add the predicted signal to thereconstructed residual signal to generate a reconstructed pixel signalof the target region; and instructions executable by the computer tostore the reconstructed pixel signal as a part of a reconstructedpicture, wherein the instructions executable by the computer todetermine motion information for acquisition of the pixel signal fromthe reconstructed picture further comprise: instructions executable bythe computer to derive one or more motion information sets out of pluralpieces of the stored motion information, each of the motion informationsets consisting of two pieces of motion information between which anyone of constituent elements differs in value; instructions executable bythe computer to select from the one or more motion information sets amotion information set to be used in generation of the predicted signalof the target region; instructions executable by the computer to directstorage of two pieces of motion information in the selected set; andinstructions executable by the computer to combine two signals acquiredfrom the reconstructed picture based on the two pieces of motioninformation in the selected set, the two signals combined to generatethe predicted signal of the target region.
 13. An image predictivedecoding method comprising: extracting, with a processor, encoded dataof motion information, the encoded data of motion information extractedout of a plurality of regions in a picture for generation of a predictedsignal of a target region of a decoding target; extracting, with theprocessor, encoded data of a residual signal from compressed data; theprocessor storing the motion information in a computer readable storagemedium; generating the predicted signal of the target region with theprocessor based on the motion information; the processor restoring adecoded residual signal of the target region from encoded data of theresidual signal; the processor adding the predicted signal to thedecoded residual signal to decode a pixel signal of the target region;the processor storing the decoded pixel signal as a part of areconstructed picture in the computer readable storage medium; deriving,with the processor, out of plural pieces of motion information stored inthe computer readable storage medium, one or more motion informationsets each consisting of two pieces of motion information between whichany one of constituent elements differ in value; determining, with theprocessor, from among the derived motion information sets, a motioninformation set used in generation of the predicted signal of the targetregion; and directing, with the processor, storage in the computerreadable storage medium of two pieces of motion information in thedetermined set; wherein generating the predicted signal furthercomprises combining two signals acquired from the reconstructed picturebased on the two pieces of motion information in the determined set togenerate the predicted signal of the target region.
 14. A non-transitorycomputer readable storage medium comprising instructions executed by acomputer for image predictive decoding, the computer readable storagemedium comprising: instructions executable by the computer to extractencoded data of motion information, the encoded data of motioninformation extracted out of a plurality of regions in a picture forgeneration of a predicted signal of a target region of a decodingtarget; instructions executable by the computer to extract encoded dataof a residual signal from compressed data; instructions executable bythe computer to store the motion information; instructions executable bythe computer to generate the predicted signal of the target region basedon the motion information; instructions executable by the computer todecode a residual signal of the target region from the encoded data ofthe residual signal; instructions executable by the computer to add thepredicted signal to the decoded residual signal to decode a pixel signalof the target region; instructions executable by the computer to storethe decoded pixel signal as a part of a reconstructed picture;instructions executable by the computer to derive, from plural pieces ofstored motion information, one or more motion information sets eachconsisting of two pieces of motion information between which any one ofconstituent elements differ in value; and instructions executable by thecomputer to determine, out of the one or more derived motion informationsets, a motion information set used in generation of the predictedsignal of the target region; instructions executable by the computer todirect storage of two pieces of motion information in the determinedset; and instructions executable by the computer to combine two signalsacquired from the reconstructed picture based on the two pieces ofmotion information in the determined set, the two signals combined togenerate the predicted signal of the target region.